CHAPTER 10
Sending Binary Data
So far, we have mainly considered the exchange of small pieces of structured information: sending and receiving of messages (with or without markup, to one or more parties), broadcasting information about various forms of presence, sending alerts using
PubSub, and the like. In Chapter 9, we delved into the Jingle framework for session
negotiation, but we focused on the structured signaling messages rather than binary
media data such as voice and video chats. In this chapter, we look at sending other
kinds of binary data, especially files. We’ll proceed in ascending order from small pieces
of binary data up to large files.
Starting Small: Bits of Binary
As we have noted, XMPP is not optimized for sending binary data. But sometimes the
data you want to send is so small and seemingly insignificant that you figure there must
be a way to send it in-band. Why should those XML zealots prevent you from sharing,
say, a small image file? Such images can be quite useful when you want to send an 
emoticon (those “smiley” and “frowny” faces that kids of all ages exchange over IM
systems), a CAPTCHA (“Completely Automated Public Turing Test to Tell Computers
and Humans Apart”) to prevent malicious bots from overwhelming your XMPP server
with new account registrations, and so on.
The XMPP developer community lacked such a method for a long time, so in 2008 they
finally defined one, called Bits of Binary [XEP-0231], or BOB for short. The approach
is simple: include a unique reference to the data in a message or other stanza, and enable
the recipient to retrieve the data from the sender if the recipient has not already cached
it.
One way to reference the data is by including an XHTML <img/> element in a message.
The  src  attribute  of  the  <img/>  element  contains  a  “cid:”  URL,  as  defined  in  [RFC
2111] and illustrated in the following example:
<message from="mouse@wonderland.lit/foo"
         to="lory@wonderland.lit/bar"
         type="chat">
137
  <body>I beg your pardon! :( Did you speak?</body>
  <html xmlns="http://jabber.org/protocol/xhtml-im">
    <body xmlns="http://www.w3.org/1999/xhtml">
      <p>I beg your pardon! 
        <img src="cid:sha1+5fb0482ba7f9a01b54a1af25060f7c783fd390af@bob.xmpp.org"/>
        Did you speak?
      </p>
    </body>
  </html>
</message>
A “cid:” URL has a special format when included in the Bits of Binary protocol: it
consists of the hashing algorithm used (here sha1), a hash of the binary data itself,
the at-sign, and the domain bob.xmpp.org (which might be used in the future for
archiving and publishing common data files).
Here we assume that the recipient does not have the data cached, so it requests the data
from the sender. This is done by sending an IQ-get that specifies the referenced cid:
<iq from="lory@wonderland.lit/bar"
    id="vh39akj2"
    to="mouse@wonderland.lit/foo"
    type="get">
  <data xmlns="urn:xmpp:bob"
        cid="sha1+5fb0482ba7f9a01b54a1af25060f7c783fd390af@bob.xmpp.org"/>
</iq>
The sender then returns the Base64-encoded data in the IQ-result (to improve readability, the data includes line breaks, but these would not occur in the stanza sent over
the wire):
<iq from="mouse@wonderland.lit/foo"
    to="lory@wonderland.lit/bar"
    id="vh39akj2"
    type="result">
  <data xmlns="urn:xmpp:bob"
        cid="sha1+5fb0482ba7f9a01b54a1af25060f7c783fd390af@bob.xmpp.org"
        type="image/png">
    iVBORw0KGgoAAAANSUhEUgAAADIAAAAyCAYAAAAeP4ixAAAABmJLR0QA/wD/AP+g
    ...
    mOjs3AAAAABJRU5ErkJggg==
  </data>
</iq>
Now the recipient has the referenced data and can render the image in the original
message (this is not unlike fetching an image to display in a web page).
The Bits of Binary approach is appropriate only for very small pieces of binary data
(typically limited to 8 kilobytes). If you have more data to send, you need to use different
methods, as we discuss next.
138 | Chapter 10: Sending Binary Data
Moving On Up: Transferring Midsize Files In-Band
Let’s say that Alice has a small camera with her when she takes her trip down the rabbit
hole. She feels compelled to share some pictures with her sister, who otherwise simply
wouldn’t believe her story. So she picks the best picture and her XMPP client prepares
to  send  all  400  KB  of  the  file  to  her  sister.  There  are  three  steps  involved  in  this
preparation:
1. Because XMPP has strict rules about what types of data it can carry, the first step
of sending binary data is Base64-encoding it, such that it consists only of alphanumeric characters.
2. Alice’s XMPP server doesn’t allow her to send stanzas that are larger than 64k, and
her sister’s server might be even more restrictive (there are good security and performance reasons for these restrictions, so Alice and her sister will have to live with
them). Because the smallest allowable number of bytes for a server-enforced maximum stanza size is only 10,000 (i.e., 10 kilobytes), Alice’s XMPP client needs to
break the 400 KB picture into smaller chunks. That is, it will chop up the data
stream that it reads from the file into a number of smaller pieces, called blocks.
3. Because her sister’s client will reconstruct the file from these data blocks, Alice’s
client needs to indicate that these blocks belong together, and that together they
form one piece of data. Her client therefore announces to her sister that she is going
to send a stream of data that is chunked into blocks of a certain size (here 4,096
bytes per block).
Let’s  see  how  this  translates  into  XMPP,  specifically  In-Band  Bytestreams,  or  IBB
[XEP-0047]. First, Alice sends a request to initiate an in-band bytestream:
<iq from="alice@wonderland.lit/rabbithole"
    id="iy2s986q"
    to="sister@realworld.lit/home"
    type="set">
  <open sid="dv917fb4"
        block-size="4096"
        xmlns="http://jabber.org/protocol/ibb"/>
</iq>
The sid attribute specifies a unique identifier for this bytestream.
After Alice sends her initiation request, her sister accepts the request:
<iq from="sister@realworld.lit/home"
    id="iy2s986q"
    to="alice@wonderland.lit/rabbithole"
    type="result"/>
Alice’s client can now start sending packets. In order to do so, it chops up the Base64encoded stream into 4,096-byte blocks (as it promised when opening the bytestream)
and puts them inside a series of message stanzas (here again the line breaks are included
only for readability):
Moving On Up: Transferring Midsize Files In-Band | 139
<message from="alice@wonderland.lit/rabbithole" 
         to="sister@realworld.lit/home" 
         id="ck39fg47">
  <data xmlns="http://jabber.org/protocol/ibb" 
        sid="dv917fb4" 
        seq="0">
    qANQR1DBwU4DX7jmYZnncmUQB/9KuKBddzQH+tZ1ZywKK0yHKnq57kWq+RFtQdCJ
    WpdWpR0uQsuJe7+vh3NWn59/gTc5MDlX8dS9p0ovStmNcyLhxVgmqS8ZKhsblVeu
    IpQ0JgavABqibJolc3BKrVtVV1igKiX/N7Pi8RtY1K18toaMDhdEfhBRzO/XB0+P
    AQhYlRjNacGcslkhXqNjK5Va4tuOAPy2n1Q8UUrHbUd0g+xJ9Bm0G0LZXyvCWyKH
    kuNEHFQiLuCY6Iv0myq6iX6tjuHehZlFSh80b5BVV9tNLwNR5Eqz1klxMhoghJOA
  </data>
</message>
For every block Alice sends, she adds a sequence number such that her sister can
detect missing blocks. The sequence starts at zero, and increments by one with each
block.
After sending the first block, Alice’s client then continues to send message stanzas,
incrementing the seq attribute by one each time:
<message from="alice@wonderland.lit/rabbithole" 
         to="sister@realworld.lit/home" 
         id="fh91f36s">
  <data xmlns="http://jabber.org/protocol/ibb" 
        sid="dv917fb4" 
        seq="1">
    dNADE1QOjH4QK7wzLMaapzHDO/9XhXOqqmDU+gM1MljXX0lUXad57xJd+ESgDqPW
    JcqJcE0hDfhWr7+iu3AJa59/tGp5ZQyK8qF9c0biFgzAplYukItzdF8MXufoyIrh
    VcD0WtniNOdvoWbyp3OXeIgII1vtXvK/A7Cv8EgL1X18gbnZQuqRsuOEmB/KO0+C
    NDuLyEwAnpTpfyxuKdAwX5In4ghBNCl2a1D8HHeUoHq0t+kW9Oz0T0YMKliPJlXU
    xhARUSDvYhPL6Vi0zld6vK6gwhUruMySFu80o5OII9gAYjAE5Rdm1xykZubtuWBN
  </data>
</message>
And so on until reaching the end of the file. After sending the last packet, Alice’s client
closes the in-band bytestream:
<iq from="alice@wonderland.lit/rabbithole"
    id="fr61g835"
    to="sister@realworld.lit/home"
    type="set">
  <close xmlns="http://jabber.org/protocol/ibb" sid="dv917fb4"/>
</iq>
Because a reply must be sent to all IQ-sets and IQ-gets, her sister’s client sends an IQresult indicating receipt:
<iq from="sister@realworld.lit/home"
    id="fr61g835"
    to="alice@wonderland.lit/rabbithole"
    type="result"/>
The picture was sent successfully.
140 | Chapter 10: Sending Binary Data
There’s one caveat with this approach: we assume that every packet we send will be
immediately received and processed by the other side. However, there are several situations where this assumption is not true: both XMPP servers could be under heavy
load, there might be a lot of traffic on your contact’s connection, or your contact might
be using a device that has minimal computing power. In this case, firing all these messages one after the other could cause you to “flood” one of the servers or your contact’s
client, as they would not be able to process all this data. To solve this problem, we can
wait until the other party has acknowledged receipt of the current packet before sending
the next one. This gives your contact as well as the intermediate servers the chance to
control the rate at which it receives the packets. To get feedback about when to send
new packets, we embed our blocks not in message stanzas but in IQ stanzas:
<iq from="alice@wonderland.lit/rabbithole" 
    id="u46sf1b9"
    to="sister@realworld.lit/home" 
    type="set">
  <data xmlns="http://jabber.org/protocol/ibb" sid="dv917fb4" seq="0">
    qANQR1DBwU4DX7jmYZnncmUQB/9KuKBddzQH+tZ1ZywKK0yHKnq57kWq+RFtQdCJ
    WpdWpR0uQsuJe7+vh3NWn59/gTc5MDlX8dS9p0ovStmNcyLhxVgmqS8ZKhsblVeu
    IpQ0JgavABqibJolc3BKrVtVV1igKiX/N7Pi8RtY1K18toaMDhdEfhBRzO/XB0+P
    AQhYlRjNacGcslkhXqNjK5Va4tuOAPy2n1Q8UUrHbUd0g+xJ9Bm0G0LZXyvCWyKH
    kuNEHFQiLuCY6Iv0myq6iX6tjuHehZlFSh80b5BVV9tNLwNR5Eqz1klxMhoghJOA
  </data>
</iq>
After a while, the response packet filters through:
<iq from="sister@realworld.lit/home" 
    id="u46sf1b9"
    to="alice@wonderland.lit/rabbithole" 
    type="result"/>
We can now send our next packet:
<iq from="alice@wonderland.lit/rabbithole" 
    id="y71fskn7"
    to="sister@realworld.lit/home" 
    type="set">
  <data xmlns="http://jabber.org/protocol/ibb" sid="dv917fb4" seq="1">
    dNADE1QOjH4QK7wzLMaapzHDO/9XhXOqqmDU+gM1MljXX0lUXad57xJd+ESgDqPW
    JcqJcE0hDfhWr7+iu3AJa59/tGp5ZQyK8qF9c0biFgzAplYukItzdF8MXufoyIrh
    VcD0WtniNOdvoWbyp3OXeIgII1vtXvK/A7Cv8EgL1X18gbnZQuqRsuOEmB/KO0+C
    NDuLyEwAnpTpfyxuKdAwX5In4ghBNCl2a1D8HHeUoHq0t+kW9Oz0T0YMKliPJlXU
    xhARUSDvYhPL6Vi0zld6vK6gwhUruMySFu80o5OII9gAYjAE5Rdm1xykZubtuWBN
  </data>
</iq>
And so on, until the entire file has been transferred.
Unfortunately, sending binary files over XMPP using the in-band bytestreams method
has some severe drawbacks. First, all binary data needs to be encoded and put inside
a message or IQ stanza. This introduces some overhead, both in speed and in generated
Moving On Up: Transferring Midsize Files In-Band | 141
traffic: besides the increased size of the encoded data, both sides need to encode and
decode the data and process the XMPP stanzas for all blocks.
However, there’s an even more severe problem than mere speed and size overhead.
Imagine someone in Europe sending a file to his friend, who happens to be sitting at a
desk right next to him. The file’s data needs to travel all the way to the sender’s server
in the U.S., over to the recipient’s server in Japan, back to the desk in Europe. Not only
does this slow down the transfer quite a bit, it also generates excessive traffic and a high
processing load on both servers. Especially when servers have many users, this quickly
becomes problematic. In practice, servers typically put a limit on the amount of data a
user can send; this practice (called “rate limiting” or “karma”) can make in-band transfers fail for larger files. Therefore, files are usually sent out-of-band, which we describe
in the next section.
In-Band and Out-of-Band
XMPP developers typically use the phrase in-band to refer to data sent
within the XMPP streams negotiated between a server and a client. By
contrast, the phrase out-of-band (or OOB) indicates that we are sending
data using a non-XMPP method, such as a peer-to-peer connection or
a channel established through a media relay.
Thinking Big: Sending Large Files Out-of-Band
When sending data out-of-band, there are two options: sending the data directly from
one entity to another over a connection that has no intermediaries or sending the data
through a third-party proxy or relay. We discuss each of these methods in turn.
Sending Data Directly
Alice’s  trip  down  the  rabbit  hole  results  in  more  than  just  one  picture.  Indeed,  she
comes home with a gazillion photos, all taken at the highest resolution (it’s good that
she always carries a few extra memory sticks with her). Since Alice’s best friend at school
is also curious, she decides to make a decent selection of her pictures, package them
up, and send them to her friend directly. Given the size of her selection, she doesn’t
want  to  send  this  data  through  a  series  of  servers,  as  we  discussed  in  the  previous
section. Instead, Alice is going to tell her friend’s client to connect to her computer
directly, and she will send the raw data through the direct connection. This approach
re-uses the SOCKS5 protocol originally defined in [RFC 1928], in particular a special
profile that is specified in SOCKS5 Bytestreams [XEP-0065].
To begin transferring her package of photos, Alice’s client first informs her friend’s
client of the network address (192.168.4.1) and port (5086) on which it can connect,
and on which she will serve the stream of data:
142 | Chapter 10: Sending Binary Data
<iq from="alice@realworld.lit/home-at-last" 
    id="p9735fg1"
    to="bestfriend@school.lit/laptop" 
    type="set">
  <query xmlns="http://jabber.org/protocol/bytestreams"
         sid="dv917fb4" 
         mode="tcp">
    <streamhost 
         jid="alice@realworld.lit/home-at-last" 
         host="192.168.4.1" 
         port="5086"/>
  </query>
</iq>
Although  this  protocol  namespace  is  http://jabber.org/protocol/bytestreams,
these are no longer in-band bytestreams, but instead SOCKS5 bytestreams.
After  her  friend’s  client  successfully  connects  on  the  given  address  and  port,  it  acknowledges Alice’s request:
<iq from="bestfriend@school.lit/laptop" 
    id="p9735fg1"
    to="alice@realworld.lit/home-at-last" 
    type="result">
  <query xmlns="http://jabber.org/protocol/bytestreams">
    <streamhost-used jid="alice@realworld.lit/home-at-last"/>
  </query>
</iq>
After Alice receives the acknowledgment, her client starts sending the data. This happens directly over a raw bytestream at the IP address and port that Alice advertised
(192.168.4.1:5086), so there is no need to chunk the stream into blocks as we did for
in-band bytestreams.
Sending Data Through a Proxy
Alice’s tech-savvy sister has installed some new security software, and now all the machines at their house are firewalled. As a result, nobody outside the house is able to
connect directly to Alice’s PC, which means that she can’t send her pictures through a
direct connection to her best friend anymore. This is where a proxy comes into play:
instead of having her friend connect directly to her machine, Alice and her friend both
connect to a third host, which transfers the data she sends to her friend.
Therefore, when Alice’s client sends the SOCKS5 Bytestreams offer, it includes the IP
address (24.24.24.1) and port (5999) of a bytestreams proxy service, in addition to
Alice’s home computer:
<iq from="alice@realworld.lit/home-at-last" 
    id="uy461vfw"
    to="bestfriend@school.lit/laptop" 
    type="set">
  <query xmlns="http://jabber.org/protocol/bytestreams" 
Thinking Big: Sending Large Files Out-of-Band | 143
         sid="dv917fb4" 
         mode="tcp">
    <streamhost 
         jid="alice@realworld.lit/home-at-last" 
         host="192.168.4.1" 
         port="5086"/>
    <streamhost 
         jid="streamhostproxy.realworld.lit" 
         host="24.24.24.1" 
         port="5999"/>
  </query>
</iq>
Her friend’s client now tries to connect to both hosts. When it discovers that Alice’s
host (the first entry) is unreachable, it tries the proxy and finds it can connect to that.
Her friend’s client therefore informs Alice’s client that it has connected to the proxy
and is waiting for data there:
<iq from="bestfriend@school.lit/laptop" 
    id="uy461vfw"
    to="alice@realworld.lit/home-at-last" 
    type="result">
  <query xmlns="http://jabber.org/protocol/bytestreams">
    <streamhost-used jid="streamhostproxy.realworld.lit"/>
  </query>
</iq>
At this point, Alice’s client connects to the proxy as well. Before sending the data over
this connection, Alice’s client first needs to activate the stream, allowing the proxy to
identify her:
<iq from="alice@realworld.lit/home-at-last" 
    id="dl4wr217"
    to="streamhostproxy.realworld.lit" 
    type="set">
  <query xmlns="http://jabber.org/protocol/bytestreams" sid="dv917fb4">
    <activate>bestfriend@school.lit/laptop</activate>
  </query>
</iq>
After this request has been acknowledged by the proxy, Alice’s client can start sending
data over the open connection.
Note that, in the previous example, we assumed that Alice specified the proxy herself.
However, it is possible for a client to automatically detect whether a proxy is available
on her server. This is done using service discovery, which we discussed in Chapter 5.
Here we look for an item in the proxy category that advertises support for the bytestreams protocol:
<iq from="alice@realworld.lit/home-at-last"
    id="o6y1g48s"
    to="example.com" 
    type="get">
  <query xmlns="http://jabber.org/protocol/disco#items"/>
</iq>
144 | Chapter 10: Sending Binary Data
<iq from="example.com" 
    id="o6y1g48s"
    to="alice@realworld.lit/home-at-last" 
    type="result">
  <query xmlns="http://jabber.org/protocol/disco#items">
    <item jid="streamhostproxy.realworld.lit" name="Bytestreams Proxy"/>
  </query>
</iq>
Alice’s client continues to “walk the tree” of entities, so it queries streamhostproxy.real
world.lit:
<iq from="alice@realworld.lit/home-at-last"
    id="r81g33fv"
    to="streamhostproxy.realworld.lit" 
    type="get">
  <query xmlns="http://jabber.org/protocol/disco#info"/>
</iq>
<iq from="streamhostproxy.realworld.lit" 
    id="r81g33fv"
    to="alice@realworld.lit/home-at-last" 
    type="result"> 
  <query xmlns="http://jabber.org/protocol/disco#info">
    <identity category="proxy"
              type="bytestreams"
              name="SOCKS5 Bytestreams Service"/>
    <feature var="http://jabber.org/protocol/bytestreams"/>
  </query>
</iq>
Voilà, a SOCKS5 Bytestreams proxy!
Unfortunately, not every server on the XMPP network deploys a bytestreams proxy.
For one thing, such proxies have the potential to use a lot of bandwidth. Thankfully,
IM users don’t send files all that often, so in practice, the bandwidth requirements are
quite reasonable. And sending data through a dedicated bytestreams proxy is much
more efficient than sending it in-band, as the proxy can be hosted on a separate machine, thus lightening the load on the XMPP daemon.
Negotiating File Transfer
Let’s take another look at the foregoing scenario. Alice starts her conversation by announcing that she is going to send some pictures:
Alice: Hey, I just finished processing those pictures from my trip down the rabbit hole!
Friend: Wow, cool! How do you want to send them?
Alice: Well, I have quite a few, so let’s send them out-of-band.
Friend: OK, works for me!
Negotiating File Transfer
| 145
Then, Alice instructs her client to start sending the files out-of-band using SOCKS5
Bytestreams. Unfortunately, her attempt fails: it seems that both Alice and her friend
are firewalled, and Alice doesn’t have access to a SOCKS5 Bytestreams proxy. Time to
revisit the plan:
Friend: Hmm, it seems we’re both firewalled. Can you send them in-band instead?
Alice: Sure, let’s try that!
Alice then instructs her client to send the files in-band, which does succeed.
What just happened is that Alice negotiated with her friend that she was going to send
a file, and told her how she wanted to send it. When finding out that the negotiated
way of sending the file didn’t work out, Alice and her friend re-negotiated how to send
the file.
However,  we  want  to  automate  these  negotiations  in  our  XMPP  clients,  instead  of
having to do this through a natural-language conversation between error-prone and
easily confused humans. In XMPP, there are two ways to do this: the older way using
the Stream Initiation protocol defined in [XEP-0095] and [XEP-0096], and the newer
way using the Jingle negotiation framework, as discussed in Chapter 9. We will explore
each in turn.
File Transfer Using Stream Initiation
The Stream Initiation (SI) protocol was the XMPP community’s first attempt at defining
a generalized media negotiation technology. Although it was never used for anything
other than file transfer, many clients support it for just that function. Over time, it is
likely that clients will transition to using Jingle for file transfer, as described in the next
section. However, we include coverage of the older method because it will probably be
in use for some time to come.
Stream initiation begins when the person who wants to transfer the file sends an offer
to the other person. This offer consists of two things: a definition of the data to be
exchanged  (via  the  <file/>  element),  and  some  possible  transport  methods  (here
SOCKS5 Bytestreams and In-Band Bytestreams), encapsulated via a Data Form inside
a feature negotiation wrapper element, as described in [XEP-0050]. The entire offer is
then wrapped in an <si/> element. (Yes, that’s a good example of XML namespaces
and XMPP extensibility in action!)
<iq from="alice@realworld.lit/home-at-last"
    id="pe72ndg9"
    to="bestfriend@school.lit/laptop"
    type="set">
  <si xmlns="http://jabber.org/protocol/si" 
      id="a0b2n44k"
      mime-type="text/plain"
      profile="http://jabber.org/protocol/si/profile/file-transfer">
    <file xmlns="http://jabber.org/protocol/si/profile/file-transfer"
          name="test.txt"
146 | Chapter 10: Sending Binary Data
The recipient now needs to decide which of the offered transport methods it prefers.
Typically, this preference will be configured into a client (as this is more user-friendly
than asking an end user to choose). Here, the preference is SOCKS5 Bytestreams:
          size="1022"/>
    <feature xmlns="http://jabber.org/protocol/feature-neg">
      <x xmlns="jabber:x:data" type="form">
        <field var="stream-method" type="list-single">
          <option>
            <value>http://jabber.org/protocol/bytestreams</value>
          </option>
          <option>
            <value>http://jabber.org/protocol/ibb</value>
          </option>
        </field>
      </x>
    </feature>
  </si>
</iq>
<iq from="bestfriend@school.lit/laptop"
    id="pe72ndg9"
    to="alice@realworld.lit/home-at-last" 
    type="result">
  <si xmlns="http://jabber.org/protocol/si" id="a0b2n44k">
    <feature xmlns="http://jabber.org/protocol/feature-neg">
      <x xmlns="jabber:x:data" type="submit">
        <field var="stream-method">
          <value>http://jabber.org/protocol/bytestreams</value>
        </field>
      </x>
    </feature>
  </si>
</iq>
<iq from="alice@realworld.lit/home-at-last" 
    to="bestfriend@school.lit/laptop" 
    id="u7yr51v4"
    type="set">
  <query xmlns="http://jabber.org/protocol/bytestreams" 
         sid="dv917fb4" 
         mode="tcp">
    <streamhost 
        jid="alice@realworld.lit/home-at-last" 
        host="192.168.4.1" 
        port="5086"/>
    <streamhost 
        jid="streamhostproxy.realworld.lit" 
        host="24.24.24.1" 
        zeroconf="_jabber.bytestreams"/>
Both clients now attempt to set up use of SOCKS5 bytestreams. In particular, the sending client initiates the SOCKS5 negotiation process by sending a set of possible streamhosts to the recipient:
Negotiating File Transfer
| 147
  </query>
</iq>
However,  as  described  in  the  foregoing  negotiation  narrative,  we  assume  that  the
SOCKS5 negotiation fails, so the friend’s client returns a <remote-server-not-found/>
error to Alice in accordance with the definition of SOCKS5 Bytestreams:
<iq from="bestfriend@school.lit/laptop" 
    id="u7yr51v4"
    to="alice@realworld.lit/home-at-last" 
    type="error">
  <error type="cancel">
    <remote-server-not-found xmlns="urn:ietf:params:xml:ns:xmpp-stanzas"/>
  </error>
</iq>
Now that the SOCKS5 Bytestreams negotiation has failed, the clients need to negotiate
the  use  of  in-band  bytestreams  instead.  However,  in  the  stream  initiation  protocol,
there is no way to fall back from SOCKS5 to in-band bytestreams, so the sender generates a new stream initiation request unconnected to the original request, this time
specifying only one transport method, in-band bytestreams:
<iq from="alice@realworld.lit/home-at-last"
    id="b93cvr3t"
    to="bestfriend@school.lit/laptop"
    type="set">
  <si xmlns="http://jabber.org/protocol/si" 
      id="a0b2n44k"
      mime-type="text/plain"
      profile="http://jabber.org/protocol/si/profile/file-transfer">
    <file xmlns="http://jabber.org/protocol/si/profile/file-transfer"
          name="test.txt"
          size="1022"/>
    <feature xmlns="http://jabber.org/protocol/feature-neg">
      <x xmlns="jabber:x:data" type="form">
        <field var="stream-method" type="list-single">
          <option>
            <value>http://jabber.org/protocol/ibb</value>
          </option>
        </field>
      </x>
    </feature>
  </si>
</iq>
The recipient has no choice but to either accept the offer and try in-band bytestreams
or reject the offer and not receive the file. Here we assume that the recipient decides to
try in-band bytestreams:
<iq from="bestfriend@school.lit/laptop"
    id="b93cvr3t"
    to="alice@realworld.lit/home-at-last" 
    type="result">
  <si xmlns="http://jabber.org/protocol/si" id="a0b2n44k">
    <feature xmlns="http://jabber.org/protocol/feature-neg">
148 | Chapter 10: Sending Binary Data
      <x xmlns="jabber:x:data" type="submit">
        <field var="stream-method">
          <value>http://jabber.org/protocol/ibb</value>
        </field>
      </x>
    </feature>
  </si>
</iq>
As  you  can  see,  in  stream  initiation,  the  only  way  to  “fall  back”  to  a  different  data
transfer method is by sending a completely new offer. This is one of the reasons why
the XMPP community is moving to the Jingle method described in the next section
(another  good  reason  is  code  reuse;  it’s  not  efficient  to  have  two  different  ways  to
negotiate sessions, and Jingle is the go-forward technology for negotiating voice, video,
and other session types).
Session Negotiation Using Jingle
The Jingle technology was originally defined for the setup and management of voice
(and more recently, video) sessions. However, Jingle can also be used to negotiate a file
transfer “session”; this usage is currently defined in [XEP-0234].
Give Me a Jingle?
The use of Jingle for file transfer is still a bit experimental at the time of
this writing. In particular, it is quite possible that more advanced methods will be defined, both for content description (e.g., to reference multiple  files  and  include  thumbnails  for  certain  file  types)  and  the  data
transport (e.g., the use of BitTorrent or ICE-TCP). Check http://xmpp
.org and the discussion lists for the most up-to-date information.
Alice starts her Jingle negotiation by offering to send a file to her friend. Upon starting
a new file transfer session, Alice’s client sends a description of the file she is about to
send and the transport she wants to use (here the SOCKS5 Bytestreams protocol). Thus,
Jingle performs the same media negotiation function as the Stream Initiation protocol
we looked at in the last section, albeit in a more modern, extensible format, as defined
in [XEP-0260]:
<iq from="alice@realworld.lit/home-at-last"
    id="fi8g376r"
    to="bestfriend@school.lit/laptop"
    type="set">
  <jingle xmlns="urn:xmpp:jingle:1"
          action="session-initiate"
          initiator="alice@realworld.lit/home-at-last"
          sid="851ba2g">
    <content creator="initiator" name="a-file">
      <description xmlns="urn:xmpp:jingle:apps:file-transfer:1">
        <file xmlns="http://jabber.org/protocol/si/profile/file-transfer"
              name="TripPictures.zip"
Negotiating File Transfer
| 149
When Alice’s best friend approves the file transfer request, her client sends a sessionaccept message to Alice:
              size="400000"
              hash="552da749930852c69ae5d2141d3766b1">
            <desc>Lots of crazy photos!</desc>
          </file>
      </description>
      <transport xmlns="urn:xmpp:jingle:transports:s5b:0">
        <streamhost
            jid="alice@realworld.lit/home-at-last"
            host="192.168.4.1"
            port="5086"/>
        <streamhost
            jid="streamhostproxy.realworld.lit"
            host="24.24.24.1"
            zeroconf="_jabber.bytestreams"/>
      </transport>
    </content>
  </jingle>
</iq>
<iq from="bestfriend@school.lit/laptop"
    id="krw927s0"
    to="alice@realworld.lit/home-at-last"
    type="set">
  <jingle xmlns="urn:xmpp:jingle:1"
          action="session-accept"
          initiator="alice@realworld.lit/home-at-last"
          responder="bestfriend@school.lit/laptop"
          sid="851ba2g">
    <content creator="initiator" name="a-file">
      <description xmlns="urn:xmpp:jingle:apps:file-transfer:1">
      <file xmlns="http://jabber.org/protocol/si/profile/file-transfer"
            name="TripPictures.zip"
            size="400000"
            hash="552da749930852c69ae5d2141d3766b1">
          <desc>Lots of crazy photos!</desc>
      </file>
      </description>
      <transport xmlns="urn:xmpp:jingle:transports:bytestreams:0">
        <streamhost
            jid="alice@realworld.lit/home-at-last"
            host="192.168.4.1"
            port="5086"/>
        <streamhost
            jid="streamhostproxy.realworld.lit"
            host="24.24.24.1"
            zeroconf="_jabber.bytestreams"/>
      </transport>
    </content>
  </jingle>
</iq>
150 | Chapter 10: Sending Binary Data
After the initiator acknowledges the session-accept message, the parties will attempt
to set up a SOCKS5 bytestream between themselves. However, here again, we assume
that the SOCKS5 negotiation fails, so the friend’s client returns a <remote-server-notfound/> error to Alice:
<iq from="bestfriend@school.lit/laptop" 
    id="od7v46xh"
    to="alice@realworld.lit/home-at-last" 
    type="error">
  <error type="cancel">
    <remote-server-not-found xmlns="urn:ietf:params:xml:ns:xmpp-stanzas"/>
  </error>
</iq>
Now that the SOCKS5 Bytestreams negotiation has failed, the clients need to negotiate
the  use  of  in-band  bytestreams  instead.  This  is  done  using  the  Jingle  transportreplace action mentioned in Chapter 9. The transport-replace message defines the
new transport to be attempted, in this case IBB:
<iq from="bestfriend@school.lit/laptop"
    id="gu72vsl9"
    to="alice@realworld.lit/home-at-last"
    type="set">
  <jingle xmlns="urn:xmpp:jingle:1"
          action="transport-replace"
          initiator="alice@realworld.lit/home-at-last"
          responder="bestfriend@school.lit/laptop"
          sid="851ba2g">
    <content creator="initiator" name="a-file">
      <transport xmlns="urn:xmpp:jingle:transports:ibb:0"/>
    </content>
  </jingle>
</iq>
Based on its supported transport methods and configured transport preferences, Alice’s
client  accepts  the  request  to  try  a  new  transport  method  by  sending  a  transportaccept message:
<iq from="alice@realworld.lit/home-at-last"
    id="yt481nf0"
    to="bestfriend@school.lit/laptop"
    type="set">
  <jingle xmlns="urn:xmpp:jingle:1"
          action="transport-accept"
          initiator="alice@realworld.lit/home-at-last"
          responder="bestfriend@school.lit/laptop"
          sid="851ba2g">
    <content creator="initiator" name="a-file">
      <description xmlns="urn:xmpp:jingle:apps:file-transfer:1">
        <file xmlns="http://jabber.org/protocol/si/profile/file-transfer"
              hash="552da749930852c69ae5d2141d3766b1"
              name="test.txt"/>
      </description>
      <transport xmlns="urn:xmpp:jingle:transports:ibb:0"
          block-size="4096"
Negotiating File Transfer
| 151
          sid="dv917fb4"/>
    </content>
  </jingle>
</iq>
Now Alice sends the file to her friend over the IBB transport as previously described.
Summary
In this chapter, we covered a lot of ground. First, we looked at a lightweight method
for sending very small “bits of binary” over XMPP. This method is mainly used to send
in-line images within XHTML messages and data forms (which we discussed in Chapter 6). Then we looked at in-band bytestreams, a way to break down a binary file into
smaller chunks so that it can be sent in an ordered sequence of multiple stanzas (either
messages or IQ stanzas). While IBB provides a reliable (if slow) mechanism when all
else fails, it’s not necessarily appropriate for sending larger files; in this case, the “outof-band” SOCKS5 bytestreams method is more network-friendly. In the future, additional methods might be defined, such as a way to initiate a BitTorrent over XMPP.
Given the existence of multiple file transfer methods, we need a good way to negotiate
which method to use, and to fall back from one method to another if necessary. The
original Stream Initiation negotiation method used in the XMPP developer community
lacks some flexibility and has never been used for anything except file transfer. Specifically, it was not adapted for multimedia session management; instead, the community
has defined the Jingle technology for more modern negotiation scenarios, and is working to migrate file transfer negotiation to use Jingle as well.
152 | Chapter 10: Sending Binary Data
